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METHOD AND ASSEMBLY FOR PROCESSING, VIEWING AND INSTALLING 
COMMAND INFORMATION TRANSMITTED BY A DEVICE FOR 
MANIPULATING IMAGES 

CROSS-REFERENCE TO RELATED APPLICATIONS 

(0001) This application claims the benefit of a priority under 35 USC 1 19(a)- 

(d) to French Patent Application No. 02 14994 filed November 28, 2002, the entire 
contents of which are hereby incorporated by reference. 

BACKGROUND OF THE INVENTION 

[0002] The present invention and embodiments thereof relates to a method 

and assembly for processing command information transmitted to means for 
processing via a device for manipulating images and, in particularly, manipulating 3D 
modelling images. The present invention and embodiments thereof also relates to an 
installation for viewing medical images in a surgical theater or examination room 
implementing the method. The present invention and embodiments thereof can be 
useful in interventional radiology or in medical applications in general, particularly in 
a real-time environment. 

10003) Peripheral input devices for manipulating 3D modelling images are 

already known. This type of peripheral device comprises a gripping element intended 
to be grasped by the user (mouse head in the case of a 3D mouse or a joystick type 
control lever), and means for forming force and/or displacement sensors which 
generate command information corresponding to the displacements and/or force 
applied by the user on the gripping element, i.e., head.. The command information is 
transmitted to means for processing which manages the 3D modelling representation 
which is displayed on a screen and which converts the command information into 
movements given in space to the representation. 

|0004| There is a growing demand for medical practitioners, such as 

radiologists or surgeons, to be able to manipulate 3D modelling images directly 
during surger>' or examination. Peripheral devices for manipulating 3D modelling 
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images known to date do not allow this in an optimum manner. In particular, such 
devices do not allow the flexibility in manipulating considered desirable when images 
are being, for example, viewed during a surgical operation. In particular, in a surgical 
theater or examination room, the radiologist or surgeon remains standing, in an 
uncomfortable position and is not accustom to manipulating an information peripheral 
device and is likely to cause a certain number of involuntary movements on the 
peripheral device. Likewise, when a sterile sheet covers the peripheral device, the 
friction from this sheet on the peripheral device can cause parasite or unwanted 
movements. 

10005] Furthermore, in the case of a peripheral device with more than three 

degrees of freedom, and especially with six degrees of freedom, it can prove 
particularly difficult for the surgeon or radiologist to carry out fully controlled 
movements of translation or movements of rotation, since such movements 
correspond in general to relatively close movements or forces on the peripheral 
device. 

BRIEF DESCRIPTION OF THE INVENTION 

j0006J An embodimenl of the disclosed and claimed invention is directed to a 

method for processing command information transmitted via a peripheral device for 
manipulaung 3D modelling images, the peripheral, device comprising means for 
manipulating by a user and means for forming sensors which detect forces and/or 
displacements on the means for gripping as a result of detected forces and/or 
displacemems generate command information, some corresponding to translation or 
zoom components, and others to rotation components for the movement to be 
conferred to a spatial representation of the 3D modelling. In a first operating mode the 
set of command information is processed to modify the displayed image by imparting 
thereto only movements of rotation in space and in a second operating mode the 
command information is processed to modify the displayed image by imparting 
thereto only movements of translation or a zoom effect. 

100071 An embodiment of the disclosed and claimed invention is also directed 

to an assembly comprising a peripheral device comprising means for manipulating 3D 
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modelling images, at least one screen on which images are displayed, means for 
processing which control the display on the screen, means for linking enabling the 
peripheral device to transmit command information to the means for processing, the 
peripheral device comprising a gripping element manipulated by a user and means for 
forming sensors which detect forces and/or displacements on the gripping element 
and generate, in terms of detected forces and/or displacements command information, 
some corresponding to translation or zoom components, and others to rotation 
components for movement to be conferred to the spatial representation of the 3D 
modelling, the means for processing comprise means suitable for using the 
abovementioned method. 

100081 An embodiment of the disclosed and claimed invention is also directed 

to an installation for viewing medical images comprising an assembly of the type 
mentioned hereinabove, the peripheral device being placed in a surgical theatre and/or 
examination room. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10009) Other characteristics and advantages of the invention will emerge from 

the following description, which is purely illustrative and non-limiting and which 
must be read with reference to the attached figures in which: 

100101 Figure 1 is a diagrammatic illustration of a peripheral device for 

manipulating images and mean for processing to which it is linked; 

[00111 Figure 2 illustrates different stages of implementation processing 

according to an embodiment of the invention; and 

{00121 Figure 3 diagrammatically illustrates a surgical iheuier and/or 

examination room that includes a 3D image manipulation peripheral. 
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DETAILED DESCRIPTION OF THE INVENTION 

10013) Figure 1 illustrates means for manipulating, such as a peripheral device 

1 for manipulating 3D modelling images and means for processing 2 to which the 
peripheral device is connected (by cable or by an RF link for example). 

100141 This peripheral device 1 can be a 3D mouse comprising a head, not 

illustrated here, which is articulated on a support at six degrees of freedom and means 
for forming sensors allowing the movemems of the gripping head to be delected by 
six components corresponding to these six degrees of freedom and to transmit 
command information corresponding to these six components to the means for 
providing the command information. 

100151 the command information is transcribed by means for processing 2 to 

give a corresponding movement to the 3D modelling image whose screen display it 
controls. An example of a 3D mouse of this type is described in US patent 4 785 180. 
The sensor of the 3D mouse is an optoelectronic sensor allowing six components to be 
detected: three translation components in three directions corresponding to three 
perpendicular axes and three rotation components corresponding to . the rotations 
around these three axes. A further example of a peripheral device is described in co- 
pending patent application filed as of even date in the name of Salazar-Ferrer et al., 
entitled: "Device for Manipulating Images, Assembly Comprising Such a Device and 
Installation for Viewing Images", (GE Docket 130600), which claims a priority under 
35 use 1 19(a)-(d) to French Patent Application No. 02 14992 filed on November 28, 
2002, the entire contents of which are hereby incorporated by reference. 

100161 In the following description, command information is illustraied by 

three parameters of translation, "x", "y" and "z",.and three parameters of rotation. "A", 
"B" and "C". The three parameters of translation "x", "y" and "z" correspond to. the 
amplitude of the components of movement along three perpendicular axes. The three 
parameters of rotation "A", "B" and "C" correspond to the amplitude of components 
of movement of rotation about these same three axes. The six parameters are 
transmitted to the means for processing which utilize the steps illustrated in Figure 2. 
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10017] Referring to Figure 2, in a first step (step 1). the means for processing 

use filtering of the micro-movements on this command information. This Hliering is, 
for example, a simple thresholding on the parameters of translation and rotation. 
Therefore, for example, a micro-movement on the mouse or more generally the 
peripheral device is avoided, due to the fact that the operator has moved the sterile 
sheet placed thereon or due to the fact that the operator has brushed the mouse 
without actually wanting to control it. 

10018) In a second step (step 11), the information of translation and rotation is 

merged. For example, a linear combination of the parameter corresponding to 
translation "x" and of the parameter corresponding to rotation "B" is determined, as 
well as a linear combination of the parameter corresponding to translation "y" and the 
parameter corresponding to rotation "A". By way of example, the parameters "x" and 
"B" are totalled and the same applies to the parameters "y" and "A". The rolalion "C" 
and translation "z" ate not merged. The means for processing 2 impose on the user a 
choice between a "rotation" operating mode and a "translation" operating mode. The 
parameters resulting from the merging step are then utilized as command parameters 
for the rotation movement, if this occurs in the "rotation" operating mode, or for 
translation movement, if this occurs in the "translation" operating mode. 

100191 Figure 2 illustrates the case where the following parameters are used as 

command parameters in the "rotation" operating mode: 

[00201 A' = A + y 

10021) B' = B+x 

100221 C' = C 

100231 In the case of ihe "translation" operating mode for example the 

following new translation parameters are used: 

[00241 x' = B + X 
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100251 y' = A + x 

(00261 z' = z 

100271 The movement of rotation or translation to be imposed by Hie user on 

the 3D image which is manipulated will be more rapid and efficacious: it will directly 
take into account for a single rotation movement or a single translation movement 
imposed on the 3D modelling image the sum of effects of translation and rotation 
imposed physically by the user on the peripheral device which is manipulated. 

[00281 In a third step (step III), the parameters or values thus obtained are 

filtered to eliminate small translation/rotation components. For example, the 
parameter A' is compared to B 72 or as well as to C 72. If A'- is less than B72 or C72 
the parameter A' is replaced by a zero value. In this way the rotation or ininslation 
components that are negligible or small relative to the other components are deleted. 
Similar comparison tests are used for the other parameters (B', C, x', y'). 

10029] The filtering of the small components allows the user to more easily 

effect a clear rotation around an axis of choice. Furthermore, the filtering treatment 
does not prevent complex rotations (respectively translations) taking into account two 
or three rotation components (respectively along a give axis) at the same lime. If none 
of the rotation components is small with respect to the other components, all of the 
components are taken into account of the final rotation component. 

100301 Steps 1-111 describes m the context of rotation and/or translation could 

be extended to other types of actions performed with a means for means manipulating. 
For example, the navigation along a reformatted cross-section (where the meiins for 
manipulating could be used to select both angles and a location of the current cross- 
section. 

(00311 Filtering tests or comparative tests other than those just now described 

for step 111 are also possible. 
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[00321 In a fourth step, (step IV), when the peripheral device is used in 

"translation" operating mode, movement along the axis "z" is interpreted by the means 
for processing as a zoom command. To prevent this zoom movement from being 
perturbed by parasite or unwanted translation movements, filtering is used such that 
as soon as it is detected that the component "z* " is not equal or different irom zero, 
the components "x* " and "y' " are replaced by zero values. In this way a non- 
perturbed and substantially clear zoom movement is effected. 

|0033] The peripheral device is particularly adapted for use in an installation 

enabling viewing of medical images in a surgical theater and/or examination room. 
With such an installation, the peripheral device 1 can be placed in a surgical theater 
and/or examination room. 

[00341 This is illustrated in Figure 3, which shows a surgical theater and/or 

examination room II, and an auxiliary control room 12 in which the calculaiion unit 
that forms the means for image processing 2 is located. 

[0035] Means for processing 2 manages the 3D image display corresponding 

to data that it receives from a medical image acquisition device (not shown) arranged 
in room 1 1 (for example, a C arm type fluoroscopic acquisition device). More 
precisely, the means 2 receives control information from the peripheral device 1 
manipulated by the surgeon or radiologist and that is located in the surgical theater 
and/or examination room 1 1, on the side of a table 19 on which the patient will be 
lying. Means 2 controls the display of 3D images on display monitors 14 and 1 5, with 
one (monitor 14) being placed in the room 1 1 and the other (monitor 15) being placed 
in the auxiliary control room (12). Cables connect the means 2 to peripheral device 1 
and to monitors 14 and 15. Obviously, other means could be provided (for e.xample 
RF transmission). 

10036] The surgical theater and/or examination room 1 1 may also comprise 

more than one monitor, for example, at least two other monitors 16 . and 17 with 
complementary images which can be connected to the image of monitor 15 using 
means 2 as a function of control instructions sent by the surgeon or radiologist 
through peripheral deyice 1. Monitor 14 in room 11 can be a flat screen monitor that 
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minimizes its size. It can be placed on a wall in room 1 1 or in an area of the room in 
which there is no or reduced risk of collision with the patient. For example, monitor 
14 may be placed facing the operating table, on the side opposite peripheral device 1 
For example, it may be adjacent to monitors 16, 17, for example, to the lefi side, and. 
if it this location is undesirable, if there is any risk of collision for the patient, to the 
right of the monitors. 

100371 In the installation for viewing or displaying an image comprising the 

above assembly at least.one means for display can be placed in an room or facility 
(12) other than a surgical theater and/or examination room (1 1). In the insiallaiion for 
viewing or displaying an image comprising the above assembly the means for 
processing (2) can be placed in room (12) or facility other than a surgical theater 
and/or examination room (1 1). 

10038] An embodiment of the method and equivalents thereof has the 

following various characteristics taken singly or in combination: 

10039] processing for filtering the rotation and/or translation components 

corresponding to micro-movements is used on the command information; 

10040] at least one rotation component and at least one translation component 

are combined and in that the combined component(s) thus obtained is (are), utilized as 
rotation component(s) in the first operating mode and as translation componeht(s) in 
the second operating mode; 

10041] one combination used is a linear combination; 

10042] a comparison is used on the. combined components intended to 

demonstrate the small components and in that in terms of the result of this comparison 
the componenl(s) thus demonstrated are replaced by a zero component: a combined 
component is replaced by a zero component when said component is less than a given 
ratio of at least one other component; a combined component is replaced by a zero 
component when the component is less, than half of at least one other component; 
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100431 in the second operating mode, after filtering of the micro-movements, 

whether the zoom component is zero or not is detected and in that when the latter is 
not zero, the other components are replaced by zero components. 

100441 Various modifications in way and/or function and/or result may be 

proposed or made by one skilled in the art to the disclosed embodiments and 
equivalents thereof without departing from the scope and extent of the invention. 
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